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ABSTRACT 

A workplace literacy project to prepare Houston 
Lighting and Power Company employees for the commercial driver's 
licence (CDL) exam was evaluated. The following four components of 
Steele's (1990) evaluation model were used to evaluate the project: 
proof of effect, judgment against criteria, critical questions, and 
valuing. The evaluation revealed a high quality program that included 
use of the Tests of Adult Basic Education for assessment, curriculum 
design and content based on assessment of job requirements, and pilot 
testing with 20 students and demonstration of the main instructional 
program. Of drivers enrolled in the class, 99 percent passed the CDL 
test. Students indicated a desire to further their learning. The 
evaluation assessed whether project objectives were me\:, namely, 
whether drivers passed the CDL exam and employees demonstrated job 
literacy skills. The following activities guided the accomplishment 
of these objectives: task analysis of job skills for literacy 
requirements; development of curriculum, a recruitment strategy, an 
instructional system and materials, and a job-specific assessment 
instrument; delivery of a pilot demonstration and demonstration 
class; evaluation; and follow-up. Consideration of critical questions 
determined that a work force literacy program was appropriate and 
that unexpected occurrences influenced project outcomes. Employees 
felt good about the learning experience and families were supportive. 
(12 references) (YLB) 
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INTRODUCTION 



"Evaluation" is a concept which usually generates feelings of 
negativity by those individuals being evaluated, particularly if an 
external evaluator is being utilized. Because of nnistakes, 
misunderstandings, and sometimes, improper attitudes about the 
role of the evaluator, the image suggested by Maanen (1979) of an 
external evaluator is occasionally warranted ". . .a steely-eyed 
^efficiency expert' who marches semi-annually into program settings 
to disrupt the established enterprise and cast a disparaging shadow 
over the efforts of a hardworking staff." 

However, the evaluator must keep in mind why evaluations are 
important - the improvement of programmatic or organizational 
activities. The evaluator*s primary responsibility is to people , not 
things, i.e., outputs, products, etc. Therefore, since there are many 
audiences involved in an evaluation, - participants, teachers, 
administrators, funding agents - the needs of all must be considered 
in the activities and final analysis. Scriven (1991) believes that "goal- 
free" evaluation is the best approach because the evaluator does not 
consider the program goals and simply lets the information flow 
from observations, conversations, and other related activities. This 
process does not enable anvone to influence the evaluation activity. 
However, Strake (1991) is more inclined to let program stakeholders 
influence the purpose and conduct of the evaluation, preferring to 
believe that evaluation can provide a service and become useful to 
the different audiences involved. 

This facilitator role is also the approach of the staff of the Texas 
Center for Adult Literacy and Learning (TCALL). Unless the 
evaluation product can be utilized for program improvement, either 
immediately or in the future, why spend the time, energy, and 
money to do it? 

Therefore, in evaluating this project, the model proposed by 
Steele (1990) in the Handbook of Adult and Contin uing Education 
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was utilized. It provides flexibility while enabling the evaluator to 
examine specific areas of endeavor. This, in turn, fosters data for 
improved decision-making by as many audiences as possible, e.g., 
**Should a similar program be funded?" "What changes should occur 
in curriculum development, teaching style(s) or recruitment?" "How 
should future contracts for a similar program be worded?" 

Principle components of the model, utilized by TCALL staff in 
previous evaluation projects, are: 

(1) . Proof of effect - determining if results are caused by 
program activities, by other events, or merely by chance. 

(2) . Jud gment against criteria - compares "what is" with what 
was planned, or "what should be" as determined by ihe 
program goals or objectives. 

(3.) Critical questions - those questions the primary 
stakeholders are inclined to ask. This aspect is more dynamic 
because in social research, the questions may change as the 
process continues. 

(4.) Valuing - determining worth. For example, how do the 
participants value their experiences? 

Initial Evaluation Project Information 

The Commercial Motor Vehicle Safety Act of 1986 requires all 
commercial motor vehicle drivers to obtain a commercial driver's 
license (CDL) by April 1, 1992 (approximately 800,000 drivers in 
Texas). All applicants must pass one or more written examinations - 
a general knowledge exam, and additional exams for special 
endorsements or restrictions. An oral exam is also available in most 
states, including Texas, but it requires the same extensive 
preparation as do the written exams. 

In March, 1990, staff of TCALL were contacted to ascertain 
their interest in evaluating the workplace literacy project. A positive 
response, plus some information about the Center, were forwarded to 
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North Harris County College (NHCC). Later, upon request from staff at 
NHCC, an evaluation plan was developed and submitted to NHCC by 
the TCALL staff for approval. Once approved, a contract was then 
signed by the appropriate officials of both institutions. 

PROOF OFEFFECr 

1. Providing a Quality Program 

Underlying all other accomplishments in any educational 
endeavor is a quality instructional program. The CDL program was 
designed with four principle components: (a) comprehensive, but 
non-threatening assessment procedures, (b) a curriculum which was 
contextually meaningful to the participants and purposeful for 
program goals, (c) pilot classes to field-test the planned instructional 
program e.g., assessment procedures, curriculum, teaching strategies, 
and other components, and (d) "power" classes for the 
main instructional program. 

(a) Initially, assessment was conducted using, the CDL Basic Skills 
Assessment developed by staff at the Institute for the 
Study of Adult Literacy, Pennslyvania State University. 
However, after using the Tests of Adult Basic Education , both 
long form and locator versions, that instrument seemed more 
appropriate to the project and was used more extensively 
throughout most of the program. This test was the most 
commonly used assessment instrument in the U.S. for adult 
literacy at the time this project was being conducted. 

(b) The curriculum design and content were based upon 

assessments of job requirements made through a "modified" 
job audit recommended by those who have acquired expertise 
in workplace literacy (Askov, et al., 1989; Drew and Mikulecky, 
1988; Wedman, J. M., and Robinson, R., 1990). The curriculum 
was developed in a contextually relevant format, again 
following the recommendations of those individuals who are 
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recognized as leaders in workforce literacy (Askov. et al., 
1989; Brown, 1990; Drew and Mikulecky, 1988, Sticht and 
Mikulecky,1984). 

(c) A pilot class was held to field-test and evaluate the curriculum 
and the classrocm teaching. These were visited by the 
evaluation team several times and observations were shared 
with the project staff. From the formative evaluations of both 
the project staff and the evaluation team, changes deemed 
appropriate for class improvement were implemented in the 
power classes which followed. 

(d) The power classes were also visited by the evaluation team 
members and again, comments about the activities were 
shared with project staff. Those are discussed elsewhere in 
this report. 

2. Completing the CDL t est successfully 

Of those drivers who enrolled in the "power class*', 99% passed 
the CDL test. This pass-rate is supurb. All adult education teachers 
would be elated to have such success or even something close to it. 
At the time the initial draft of this report was being developed, the 
student who had completed the power learning course and had not 
passed the CDL test was still pursuing that quest for passing the test. 

3. Furthering one's learning 

One of the goals of the NHCC staff was to improve not only the 
drivers' competency in their driving abilities, but to also develop 
within them the desire to continue their education in whatever area 
they wished to obtain more knowledge. Realizing that individuals 
who are functioning below the high school equivalency level have 
needs which they consider more important than continuing 
education, this goal was considered "chancy", at best. 



When the evaluation staff visited the power class for the 
second time in May, 1991, one individual who had already passed 
the CDL test had returned to class to "learn more". When 
interviewed by the evaluation team, he indicated that he had found 
"learning" to be a pleasant experience and he wanted more. He was 
beginning his quest by becoming more informed about driving and 
planned to learn more about other aspects of his job. During the next 
few weeks, several other drivers also returned to the class after 
passing the test for the same reason(s). 

While interviewing other students in the power class, the 
TCALL staff found several who also wanted to take other classes if 
offered the opportunity. They had also discovered that they, indeed, 
could learn new "things" and this experience had motivated them to 
want to learn more. Telephone interviews with other students 
indicated that this desire did exist. The kinds of knowledge wanted 
by those interviewed ranged from job-related information to skills 
for coping with insurance and tax information. Similar responses 
were obtained by the teaching staff during informal discussions with 
students as reported to the evaluation team. 



JUDGMENT AGAINST CRITERIA 



The primary criteria which guided the development of this 
project were the following objectives: 

1. After instruction, a driver will be able to pass the driver 
certification test required for licensing by the Texas 
Department of Public Safety according to the criteria 
established by the Texas Department of Public Safety and 
the Texas Department of Transportation; and 

2. Presented with a written, oral, or simulated representation 
of a situation which requires a demonstration of literacy skills 
in combination with Job skills associated with the job of a 
vehicle driver, the employee will be able to demonstrate 
competency according to the standards set by the Houston 
Lighting and Power Company (HLPC). Specific activities 
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were developed to guide the progress toward these objectives. 
Each of those activities is discussed in regard to its degree of 
accomplishment. 

1. Task analysis of job skills for literacy requirements 

Through a cooperative effoc^ between staffs of both NHCC and 
HLPC, a task analysis, or "modified** job audit was conducted. 
Data from this audit were the basis for the curriculum which 
was developed for the project. (For example, one of the skills 
identified as needed by the drivers included . . . "interpreting 
material catalogs, work orders, street guides, and related line 
maps. . 

2. Development of curricula 

The curriculum for this project was based upon two principal 
products. The R.OA.D. (Real Opportunities for Advancement and 
Development) to Success, developed by staff at the Institute for 
the Study of Adult Literacy, Pennsylvania State University, is a 
basic skills course containing interactive computer-based 
courseware and print-based materials to assist drivers in 
developing reading skills in preparing for the CDL test. That 
curriculum was developed in cooperation with the 
Pennsylvania Department of Education, the American 
Federation of State, County, and Municipal Employees, and the 
Pennsylvania Department of Transportation. An evaluation of 
the course-of-study (Brown, 1990) concluded that the content 
effectively increased the job-related skills of transportation 
employees while increasing their abilities to comprehend the 
manual and pass the CDL examination. In addition, 
participation in the program increased workers' confidence in 
their abilities to pass the examination. The program has been 
utilized in many states by several organizations and agencies 
for CDL training classes. 

The second product was the Texas Commercial Motor Vehicle 
Drivers Handbook which was developed to assist drivers in 
preparing for the CDL test. It contains seven sections, i.e., 
driving safety, transportation, cargo safety, air brakes, etc., 
which contain key points throughout and provide relevant 
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study and review questions at the end of the chapters. It is 
comprehensive in design,- easy to follow, and utilizes practical 
. examples, including illustrations to emphasize key points. 

3. Development of a recruitment strategy 

The project staff had anticipated that there would be 
opportunities for the drivers to participate in classes during 
**company time'\ That anticipation was one of the reasons 
for the target power class enrollment figure of 200 drivers 
being included in the project proposal. However, when that 
requirement was not implemented by the company, 
recruitment efforts by the project staff and by some drivers in 
the program were not sufficient to achieve the goal of 200 in 
both the pilot and power classes. 

4. Development of instructional system and materials 

During the summer of 1990, TCALL staff periodically visited 
the literacy project staff to (a) learn as much about the project 
activities as possible, (b) observe the staff in their curriculum 
development procedures, and (c) ascertain what problems, if 
any, were affecting preparations for initiating the first classes. 
Interactions with project staff, both individually and in groups, 
enabled both groups (NHCC and TCALL) to reach clarity about 
proposed project acHvities, including staffing for the pilot 
class, recruitment of students, and location of classroom sites. 

Teachers were employed and trained by the NHCC project 
staff. Principles of adult education were followed in the 
training in order for the teachers to learn not only the kinds 
of materials they would be teaching, but also how to teach 
adults trying to learn in a stressful situation. 

Materials were developed specifically for the HLPC employees 
to pursue their studies to obtain the CDL. Materials contained 
job-related information based upon the "modified** job audits 
conducted by the project staff. They were reviewed by staff of 
both NHCC and HLPC before being included in the instructional 
program. 
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5. Development and administration of a job-specific assessment 
instrument 



The level of knowledge of each student/driver was assessed 
by a formal testing procedure (described previously), plus 
an interview. This assured that any data pertinent to the 
student's ability to begin studying at a particular level of 
competence which might be missed by the formal test, 
would be ascertained through the informal interview. 

6. Develop and implement a career counseling and management 
system for participants 

This objective was eliminated through negotiations with 
the funding agency. 

7. Deliverv of a pilot demonstration class for 20 employees 

The pilot class began in August, 1990. The TCALL evaluation 
staff observed the second meeting of the pilot class. The 
classroom was well lighted, roomy, contained tables and chairs 
for the students, and had brightly-colored safety signs on the 
walls. The project director introduced the teacher to those who 
had not attended the first class and explained the purpose of 
the class again for those individuals. There were about 20 
students in attendance. 

The teacher established good rapport with the participants and 
used a question-and-answer format to elicit their 
participation. Most became actively involved in the discussions, 
although a few chose not to do so. Students were encouraged to 
help each other when assistance seemed needed. In general, 
the TCALL staff felt comfortable with the class setting and the 
instructional approach. However, they indicated to the project 
director that the pace seemed somewhat slow because most of 
the students were staying ahead of the teacher in their 
workbooks during the evening class. There was no way, from 
one observation, to ascertain if some students would be unable 
to maintain the pace even though others apparently were 
staying ahead of the instruction. 
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The following month, another class visit was made by TCALL 
staff and several changes had occurred: 

a. Attendance had dropped somewhat because some of the 
participants felt they could proceed "on their own" faster than 
they could in the class. 

b. The class had been divided into two ability groups to provide 
more individualized instruction and to enable students with 
similar ability-levels to participate in the same group. This, in 
turn, seemed to foster much more information exchange among 
the students in each group. 

c. No name cards were needed because everyone knew 
everyone else in the class and a friendly, informal atmosphere 
seemed to have been created. 

d. In addition to the teacher asking questions about the lesson, 
students were observed interacting for clarification of different 
topics and content. Ample time was given for correct answers 
to be recorded in the workbooks and questions were repeated 
until it seemed that everyone had attained the desired content. 

e. Although attendance was lower, those in attendance were 
genuinely concerned with learning the material being 
discussed. Both teachers seemed sensitive to the needs of their 
students and the students realized that there were no "dumb 
questions" in the classes. 

8. Formative and summative evaluations and revisions of the 
model 

Upon completion of the pilot class, the NHCC project staff, plus 
one member of the evaluation staff, evaluated the model and 
made the apparent needed changes in both content and 
teaching strategies. These, then, were implemented with the 
"power" classes which began within a short time. 

9. Delivery of a demonstration class to the remaininji 18 Q 
employees in need of training 
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In March» 1991, the regular class with the revised curriculum 
from the pilot class was visited by evaluation team members. 
The classroom site had changed and although the "physical" 
quality had decreased in regard to lighting and physical 
surroundings, i.e., walls and ceiling, the students seemed to be 
comfortable. The revised cuniculum was basically designed as 
"self-paced". Participants were divided into two groups - 
those who could read the materials in the Texas Commercial 
Motor Vehicle Drivers Handbook and those who were at lower 
reading levels. Instructors were working with both groups - 
the higher-reading group was seated at tables in a "U-design" 
facing the front of the room, and the lower-reading group was 
seated around tables in another part of the room. 

At the beginning of class, the instructors expressed to the 
students their concerns about the students* apparent desire to 
"hurry through the lesson and get out the door." They felt that 
this could lead to a possible tendency to also hurry on the CDL 
test and result in mistakes which would lower one's score. 
After that discussion, a film about mountain driving was 
shown, followed by a discussion of the major concepts 
contained therein which were then put on the chalkboard for 
emphasis. One instructor was well aware of not only the 
important concepts, but also the "slang" words used by the 
drivers and was able to relate those in a meaningful context to 
the students. In addition, he was able to emphasize those 
words and concepts which would probably be found on the 
test. It was quite apparent that the drivers found this to be 
reassuring as they moved through the content of the study 
handbook. 

The goal of 180 additional employees was not reached for the 
power classes. "Company time" was not provided for employees 
to attend classes as anticipated by the project staff. However, of 
the 1200 employees in the target pool, 1190 were assessed by 
both the TABE and the CDL test in January. In addition, more 
than 500 other drivers were enrolled in a one-day training 
session in "test taking" skills. The project staff is to be 
commended for its commitment to reaching as many 
employees as possible even though their original plans for 
recruitment were not supported by the company. 
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10. Develop continuation plan and lonp ranee tracking of 
participants 

Follow-up was conducted for 90 days after the project was 
completed to ascertain if further services could be provided by 
NHCC staff. In addition, materials have been given to HLPC staff 
for use with other employees who may need training for the 
CDL test. Consultant services are also available to the company 
from NHCC staff when needed. 

CRITICAL QUESTIONS 

1. Was the philosophy underlying the project 
appropriate for the clientele being served? In general, 
orientations, or philosophical approaches to literacy on the workplace 
follow two themes; worksite literacy, in which workers or potential 
workers are trained in skills for specific jobs only, and workplace, or 
workforce literacy where workers are trained for speciHc jobs, but 
also pursue basic literacy skills to enable them to transfer to other 
jobs or tasks in the worksite (Chase, 1990; Spradling, 1990). The 
latter was the underlying approach for this project and the 
evaluation team definitely supports this choice. Although faster gains 
seem to occur in learning which is worksite oriented, when new tasks 
must be met, those workers must be retrained for specific skills as 
they are needed. In a workforce literacy program, knowledge and 
skills learned which extend beyond those required for specific tasks 
can often be transferred and utilized in the new work situation. 
Although these programs may require more time and support 
initially, the learning is more useful for much longer periods of time 
and prepares the worker to be more functional in the workforce. 
According to Bernstein (1988), this is the best approach if the United 
States is to be able to remain competitive in the international 
workforce arena in the future. The billions of dollars being spent tp 
retrain millions of displaced workers must be utilized more 
efficiently to successfully compete with other countries. 
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One member of the evaluation team participated in a worksite 
literacy program for preparing drivers for the CDL test in another 
state. The instructor in that program was very clear in explaining 
that she was not interested in the literacy needs nor the transfer 
skills of the participants. Her purpose was to prepare drivers to pass 
the CDL examination either in writing or orally. She taught strictly to 
the test and believed she was providing the service the company 
wanted without any frills or additional activities. Discussing the 
merits of workforce literacy with her was a moot point. She was not 
concerned that drivers may pass the test without having tbi ability 
to read or interpret important highway signs or markers. Although 
she recognized the importance of truck drivers being able to read, 
she rationalized her approach to the testing issue by emphasizing 
that she considered saving jobs more important than literacy, safety 
concerns notwithstanding. In the opinion of the evaluation team 
member who observed that approach, the program designed and 
implemented by the NHCC staff was more professional, more 
futuristic, and more related to the national goals of developing a 
competitive workforce through a well trained and prepared body of 
available workers. 

2. What unexpected occurrences influenced the 
outcomes of the project? 

The most influential unexpected occurrence was the 
reorganization and change of leadership at HLPC during fall, 1990. 
Virtually all of the "contact" people for the NHCC project staff 
changed, resulting in several delays, the most critical being the 
assessment schedule which was delayed for several months. This, in 
turn, delayed all other project activities which were dependent upon 
the assessment procedures. 

Another important occurrence was the change of project 
directors at NHCC several months into the project. When people are 
employed, there is no assurance that changes in their lives will not 
prohibit their continuing in their employment roles. Those conditions 
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do occur and must be dealt with by institutional leaders. In the case 
of this project, the new director had already been serving as a 
consultant to the project and her background, professional 
preparation, and experience made her a logical choice for the 
position. Fortunately, she was both willing and able to assume the 
position and gave excellent leadership to the project. However, even 
under ideal conditions, changes in the leadership of educational 
activities often cause some delays in implementing program 
priorities. Ii. this project, those were minimal. 

A change in the size of the target population, determined by 
the company, also influenced progress. The original agreed-upon 
population - 2000 drivers - was reduced, for company budget 
purposes, to 900. It was subsequently raised to 1200. In essence, the 
project staff, for a given period of time, was unable to determine the 
exact size of the available target group. 

Another occurrence was the unexpected lack of availability of 
the program (classes) on ompany time" as had been anticipated by 
the project staff. This reduced enrollments in the power classes 
below what had been expected and resulted in some disappointment 
by the project staff. 

VALUING 

1. Many of the employees who pa rticipated in the CDL classes felt 
gnnd about what they had learned about themselves as well as 
what thev had learned about driving . As one individual 
indicated in a follow-up interview, " I really feel good about 
accomplishing something. It was something I had set out to do 
and I accomplished it." Others indicated that they weren*t 
accustomed to setting a learning goal and then completing it. "I 
couldn't have done it without you all (the teaching staff). 
There's no way I could have done it." 
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2. Some empl oyees indicated that they would attend other 
classes if HLPC would offer them . However, almost all prefer 
job-related classes which would benefit them in their present 
work or prepare them for future jobs within the company. 
In addition, classes to prepare them for the GED test were 
desired by several of the students who had just completed 
their CDL test. Some who had just passed the test were not 
interested in any further classes of any kind. 

3. The families of those who com pleted the program were very 
su pportive of the learning efforts . In the interview 
immediately following the CDL test, all of those employees who 
participated in the classes were anxious to tell their families 
about their success on the test. They were also proud to share 
their success with co-workers and indicated that co-workers 
were just as anxious to learn about their test results as the 
students themselves. Family celebrations were held for some of 
those who passed the test after participating in the classes. 



CONCLUSIONS 

1. In general, the objectives of the project were attained by the 
project staff. 

2. The "pass" rate on the CDL test by those employees who 
participated in the power classes exceeded the expectations of 
project staff and evaluation teams members alike. 

3. Good adult education principles and practices were 
implemented throughout the project, resulting in successful 
performances by the employees on the CDL test, plus positive 
attitudes toward further education by those same employees. 
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RECOMMENDATIONS 



1. The staff at NHCC should pursue further programs in workforce 
literacy because of the knowledge learned through their 
experiences in the project. 

2. Professionals in other institutions and organizations should 
avail themselves of the information contained in the final 
report from this project before attempting to implement 
similar programs in their own communities. 
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